Evaluating Long Term Impacts Of Internet Training In An Academic Setting: The Romanian Internet Academy

The diffusion of Information Technology (IT) depends on good training--but the results of training are difficult to measure. Whether the focus is the student, the businessman or woman, mechanic, manager or researcher-the simple act of delivering training in a specific technology has little effect-it must be leveraged through careful practice and repeated use. Most of the research on Internet training indicates that success measurements for such courses are primarily based on measures of like or dislike of trainers, facilities, class notes, bandwidth, etc.--subjective satisfaction. Subjective satisfaction measurements are poor predictors of achievement. They usually show that persons like to learn how to use the Internet but disclose nothing about the students use of the learned skills. Being satisfied with an experience does not predict that the results will be beneficial in long-term use. IBM, the World Bank and many other organizations have concluded that it is far more important to evaluate what really happened after training (outcomes) rather than simply measure the immediate perceptions of course participants (outputs). This study, a sample of a larger work to be published in late 1998, attempts to integrate the measurement of results (outcomes) into the problem of assessing Internet training for professors in the Romanian Academy of Sciences. What sets it apart is the concentration on actual results in terms of careers, confidence and earning power, instead of the more simplistic measures normally employed. OVERVIEW The setting for this study is the Romanian Academy of Sciences, Bucharest, Romania. We feel that the course we describe is one of the more rigorous of its type in the region-but but it is certainly not the only one. For instance, the Kiev Regional Office of the Eurasia foundation in 1995 offered grant opportunities for facilitating Internet Communications in Ukraine, Belorussia, and Moldova. The University of Moldova proposed the development of an Internet site to train users in navigation on the Internet. The successful 1996 program trained over 150 lecturers, scientists, postgraduates, and students over a 3-month period in electronic mail, tools for remote access and searching, and general Internet resources. (R. Dumbraveanu, 1997). A study of researchers in Evaluating Long Term Impacts Of Internet Training In An Academic Setting http://www.racai.ro/ICASIT/study.html 1 of 8 9/6/2008 15:05 Chile (Ruth and Goet, 1993), also found good training to be essential to Internet technology diffusion, but the focus was country wide, not specific to a specific location. The Romanian Academy of Sciences is a large, multifaceted alliance of academics representing all fields of study, from drama to engineering. It was founded in 1866 in Bucharest and included members from all the Romanian provinces: Muntenia, Moldavia, Transylvania, Bassarabia, Banat, Maramures, Bukovina and Macedonia. It functioned much like other European science academies until the Communist takeover. In 1948, the traditional Academy was dismantled, its constitution and by-laws were modified, and a state Academy was installed by directly nominating a number of full and corresponding members. It functioned successively under the name of the Academy of the Romanian People's Republic and the Academy of the Socialist Republic of Romania. Over 100 former members were not included into this new institution, especially in philosophy and history. A number of leading communist names were included, many being individuals with no academic credentials. The Academy was transformed into an ideologically dominated institution . December 1989 has marked the rebirth of the Romanian Academy. In addition to the renewal of many members of the Academy expelled in 1948, some 130 new corresponding and full members were elected as well as 31 honorary Romanian members. Since its foundation in 1866 until now, the Romanian Academy has elected over one thousand members from representatives of the scientific, cultural, artistic, ecclesiastical, political and military life of the country. Outstanding personalities of science and culture from different countries of Europe, Asia and America have been included among its members. Since 1869, the year of the first elections of foreign members, 434 persons from 33 countries have been elected by the highest forum in Romania, 21 of them being Nobel Prize winners Romanian Academy: Increasing Its I.T. Capacity The Romanian Academy has three regional branches (Cluj, Timisoara and Iasi). It is organized into 14 sections: Philology and Literature, Historical and Archaeological Sciences, Mathematical Sciences, Physical Sciences, Chemical Sciences, Biological Sciences, Economical Sciences, Engineering Sciences, Agricultural and Forestry Sciences, Medical Sciences, Economic and Law and Sociological Sciences, Philosophical and Psychological and Pedagogical Sciences, Arts and Architecture and AudioVisual and Information Science and Technology (the newest section) and has 66 institutes and centers , with more than 2600 researchers. The Internet Laboratory Being mostly oriented on the humanities and theoretical sciences, the Romanian Academy had little contact with Internet technology until recently. In August 1994, after three years of discussion, the Section for Information Science and Technology, with the support of Academician Mihai Draganescu -at that time the President of the Romanian Academymanaged to establish a small research center focused on basic research in Artificial Intelligence: Romanian Academy Center for Advanced Research in Machine Learning, Natural Language Processing and Conceptual Modeling-RACAI for short. With a modest endowment, consisting of three SUN3/50 and a MAC II donated by the IDSIA-Lugano, plus a PC486, the new Center established an Internet node in the "Casa Academiei", a site that hosts 22 research institutes and centers. Although the initial speed was slow (14.4 kbs on a half duplex leased line), it was a beginning. With funds from several European projects the center partnered with Ilp-Net, Multext-East, Elsnet-Goes-East the infrastructure gradually improved. By early 1995 a powerfully configured SUN4/85, a Pentium, a Cisco router, and 2 fast modems were added. Plan for the Internet Academy Based on discussions with George Mason University and the Andrew W. Mellon Foundation New York, RACAI established the Open Internet Laboratory based on its own network. The Mellon grant added five new SunSparc stations (1 Sun Sparc-20, 4 Sun Sparc-4/85) and a Macintosh. Sun Microsystems provided additional support (in the form of 5 Sun Sparc Classic). The Romanian Academy provided the lab space, bought the furniture, and hired a lab and network administrator. As noted before, Evaluating Long Term Impacts Of Internet Training In An Academic Setting http://www.racai.ro/ICASIT/study.html 2 of 8 9/6/2008 15:05 improving the hardware infrastructure was necessary in order to give Internet access to more scientists, but a prerequisite was to build up a continuous education program on Internet technology, specifically designed for non-computational academic community. In December 1995, RACAI started a series of on-line free introductory courses on network communication aimed at a 200-person segment of researchers from the institutes of the Romanian Academy. Special emphasis was placed on researchers from the humanities. This experiment, known as the RACAI Internet School (Tufis, 1997), had an overwhelmingly positive response, and was extended for another two years (1997-1998) with a horizon of 400 more scientists. Recently the Soros Foundation also added funds to this effort. At present, the Open Internet Laboratory , which serves as a classroom, a research laboratory and a communication node for Romanian Academy (to which several new local networks have been or are in process of being connected), is equipped with 3 Cisco routers, 10 modems for dial-up services, 11 Sun Sparc stations (plus 3 rarely used Sun 3 machines), 1 PowerDuo Macintosh, 3 Pentiums, 2 network laser printers and 2 back-up units. The Internet access is ensured by RACAI through two full duplex leased-lines connected to the national academic Internet service provider, being integrated in the National Computer Network (RNC): a 128 kbps connection to CNSCC, and a 33 kbps connection to ICI (RNC operator). The latter connection will be soon replaced by a 512kbps line. Due to the gateway connections between RNC and all other Internet service providers, any user connected to a national or regional Internet node (regardless of the service provider's identity) will reach RACAI server(s) through local lines only. The RACAI node is directly connected to the main node of RNC, which in turn has direct connections to the main nodes of the high-schools network (Soros foundation and Mellon-funded), and of KAPPA (network service provider for the Romanian Government and for many domestic users), which is widely accessible for certain categories of Internet users. The RACAI server may also be accessed by switched line and up to 10 users can have simultaneous accesses (hunting system). The lab, maintained and supported by RACAI, functions as a shared resource to which the researchers from the Romanian Academy have free access. A significant number of those taking the Internet courses become regular users of the Internet lab. Since september 1997, the RACAI network has become an Internet backbone node. Besides the Mathematical Applied Institute, 5 new Academy institutions from Casa Academiei (Center for Mathematical Statistics, Institute for Quality of Life, Publishing House of Romanian Academy, Institute of Sociology, Institute of Linguistics) own networks. connected to Internet by RACAI. Internet School Curriculum The Center organizes courses every two weeks in the Internet Lab to introduce the main network communication tools to the researchers from the Romanian Academy, many of which are from humanistic domains and have not used computers before. As of 1998, two kinds of courses are implemented at the Center, one for beginning students (Part I) and one for advanced students (Part II). Part II is reserved for students who have taken the first part or who are accustomed to the relevant Internet applications. In Part II, handbooks are not given to students though they are given information on a floppy disk drive. To see this information, visit http://www.racai.ro/curs-nou. You may also view some of the course lecture notes at http://www.racai.ro/manual. Each course lasts 3 days (8 hours/day) and is practice-oriented, the students learning to effectively use the Internet tools. The first half of each day is an interactive presentation of the day's topic by a lecturer assisted by a supervisor. The students are guided through several lessons by their assigned computer. The second part of each day is for individual exercises, the students being assisted by the lecturer and the supervisor. The instructors (lecturers and supervisors) are researchers from the Center, most of them holding a joint appointment with the Polytechnic University of Bucharest and having extensive teaching experience. Comprehensive lecture notes (of about 180 pages) have been prepared, specially targeted to first-time users of computers and Internet. The contents of the lecture notes are: Introduction 1. Basic notions for understanding and use of computer networks 2. Electronic mail 3. FTP electronic transfer of documents 4. Searching for information in the Internet 5. Netscape 6. Harvest 7. Hot Java 8. Glossary 9. Bibliography 10. Evaluating Long Term Impacts Of Internet Training In An Academic Setting http://www.racai.ro/ICASIT/study.html 3 of 8 9/6/2008 15:05 Given the specific profile of the students, with no (3%) or very little (85%) previous experience on computer use, but with very specific interests in their own domains of activity, the courses were dynamically adapted so as to diminish the risk of frustration and to stimulate scientific curiosity. There were no enrollment restrictions. The course, all taught by RACAI staff, had a public announcement from the management of the institutes and has been advertised in mass-media (radio, press: the "Academica", a monthly review). It is free and takes place in the Romanian Academy House, where the majority of the institutes in Bucharest have their sites; consequently, for the researchers living in Bucharest there were no impediments to participation. The classroom, equipped with 10 Sun workstations, accommodates 20 students at a time. A WWW site (http://www.icasit.org/romania /study.html) has been established to show the center and the labs. (Harter, 1997) Besides the lecturer, a supervisor offers technical assistance and additional explanations on practical assignments. The lecture notes are distributed as an auxiliary teaching material. The text is rendered in a readerfriendly form and does not require previous knowledge in the field. More than 400 participants attended the courses organized in 1996 and 1997.At least 200 more students will take the courses in 1998. After graduation all the students are granted unconditional access to the Internet Laboratory facilities with follow-up assistance and consequently, most of them became regular users of the Internet Laboratory. The majority of the attendees were from the humanistic institutes (Linguistics, Psychology, Philosophy, Art History, Literary History and Theory, Geography, Sociology, Legal Research); the others were from positivist sciences (Physical Chemistry, Applied Mathematics, and Mathematical Statistics) TABLE 1 Subjective Satisfaction Scores for Internet Courses Taught at the Romanian Academy 1995-1997 Marks: Questions: 1= unsatisfactory Q1. Was the instructor well prepared for the course? 3= adequate Q2. Was the course material presented in an organized way? 5= remarkable Q3. Have you been intellectually implied during the lesson? Fundamental concepts E-mail FTP Xarchie Netscape Harvest Q1 4.58 4.69 4.61 4.67 4.71 4.68 Q2 4.42 4.52 4.26 4.42 4.44 4.40 Q3 4.15 4.17 4.08 4.23 4.20 4.18 TABLE 2 Age Distribution of Respondents Frequency Percent Cumulative Percent